arXiv:1508.05835v2 [math.PR] 6Jun2016 


AV2b.tex 


An HJB Approach to a General 
Continuous-Time Mean-Variance Stochastic 

Control Problem f 

G. Aivaliotis^ & A. Yu. Veretennikov^ 


June 7, 2016 


Abstract 

A general continuous mean-variance problem is considered where the 
cost functional has an integral and a terminal-time component. The problem 
is transformed into a superposition of a static and a dynamic optimization 
problems. The value function of the latter can be considered as the solu¬ 
tion to a degenerate HJB equation either in viscosity or in Sobolev sense 
(after regularization) under suitable assumptions and with implications with 
regards to the optimality of strategies. 
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1 Introduction 


Mean-variance optimisation problems have been established as a dominant 
methodology for portfolio optimisation. Markowitz IfTOll introduced the single¬ 
period formulation of the problem in 1952. It was not until the beginning of the 
new century, however, that dynamic mean-variance optimisation by means of dy¬ 
namic programming received much attention, mainly due to the difficulties that 
the non-markovianity of the variance introduced to the problem. As an alternative 
to dynamic programming, the problem was solved using martingale methods (see, 
e.g., Bielecki et al. BUl) or risk-sensitive functionals (see, e.g., Bielecki et al. |[5l), 
whose second order Taylor expansion has the form of a mean-variance functional. 

A major advance in the theory for mean-variance functionals came by em¬ 
bedding the original problem into a class of auxiliary stochastic control problems 
that are in Linear-Quadratic form. This approach was introduced by Li and Ng 
[[8| in a discrete-time setting, while an extension of this method to a continuous¬ 
time framework is presented in Zhou and Li IfTdll . and further employed Lim 0. 
This approach leads to explicit solutions for the efficient frontier under some con¬ 
straints imposed the optimisation problem (they assume that the cost function is 
a linear function of the controlled proccess). Wang and Forsyth [[T3ll design nu¬ 
merical schemes for auxiliary linear-quadratic problems formulated in [[T4l and 
construct an efficient frontier. In IfT^ . Tse et al. show that the numerical schemes 
designed in ifTSll provide indeed all the Pareto-optimal points for the efficient fron¬ 
tier. 

Aivaliotis and Veretennikov [[U propose an alternative methodology that em¬ 
beds the mean-variance problem into a superposition of a static and a dynamic 
optimisation problem, where the latter is suitable for dynamic programming meth¬ 
ods. Solutions in the spaces of functions with generalised derivatives (henceforth 
called Sobolev spaces) are obtained through reqularisation. A further extension of 
this method is presented in [|3 where the viscosity solutions approach is followed. 
In the latter, each of the functionals either depending on the terminal value of the 
controlled process or on the integral from time 0 to time T of the controlled pro¬ 
cess are considered but separately. This approach does not in general provide any 
explicit solutions, but is geared towards numerical approximations that are proven 
to work efficiently. One advantage of the proposed methodology is that the prob¬ 
lem can be solved for a pre-determined coefficient of risk aversion. For the LQ 
approach, the whole efficient frontier has to be traced and then optimal strategies 
can be assigned to different coefficients of risk-aversion. 

Let us consider a d-dimensional SDE driven by a d-dimensional Wiener pro- 
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cess (Wt, J^t, t>0) 

dXt = b{at,t,Xt) dt + a{at,t,Xt) dWt, t>tQ, Xt^ = x. (1) 

We will speeify the properties of the eoeffieients b and a later, depending on the 
approaeh we use. The strategy {at, to < t < T) may be ehosen from the elass A 
of all progressive measurable proeesses with values in A C sueh that A ^ 
bounded and elosed. We will use the standard short notation where the depen- 
denee of X on the strategy, initial data x and to is shown by ^ with respeet to 
expeetation; the full notation would be 

Consider a eost funetion / : x [0,T] x —)■ M. The eost from 

time to to T for a certain path of a process © and strategy a & A will be 
f{as, s, ds. At the terminal time T, we will eonsider a “final pay¬ 

ment” <f)(Ar). Thus the expeeted eost from to to T for a eontrol strategy a ^ A 
will be 



(/: 



In finaneial applieations the standard terminology of “eost” has often a positive 
meaning, representing some funetion of portfolio returns or eashfiows. Thus in 
the mean-varianee eontrol problem, one aims at maximising the expeeted eost 
while penalizing for varianee (that represents risk).The value funetion in this ease 
will be 



( 2 ) 


In this paper we are going to eonsider solutions of this problem both in Sobolev 
spaees and in Viseosity sense and diseuss the optimal strategies. The eontribution 
of this paper is threefold: we eonsider a general integral and terminal time pay¬ 
ment funetional, we relax the assumptions on boundedness of the eoeffieients and 
eost funetions and we diseuss the optimality of strategies in different settings. 

With regards to the funetional, the problem without a“final payment” has 
been diseussed in [[T]| with solutions in Sobolev spaees and the problems with 
either only integral funetional or only “final payment” were diseussed in O using 
Viseosity solutions. It is elear, however, that the solution to problem ©I eannot be 
derived as a eombination of the previous two eases due to the supremum involved. 
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When looking for solutions in Sobolev spaces, we need to use regularisation 
(similar to [[U) as we cannot relax the non-degeneracy assumption (which is not 
needed for Viscosity solutions). We do however relax the assumptions regarding 
the boundness of the drift and diffusion coefficients as well as of the cost function 
/ in comparison to the assumptions used in |[T1. Finally we show that regularisa¬ 
tion results into £—optimal strategies, whereas a verification theorem for Viscosity 
solutions is only attainable under strict boundness assumptions, which are not ful¬ 
filled in our context. 


2 Mean-Variance Control 


The goal of this paper is to maximize a linear combination of the mean and vari¬ 
ance of a payoff function that involves both an integral and a final payment. The 
value function Q presents a non-markovian optimisation problem. This is due to 
the time-inconsistency of the variance term due to the square of the expectation 
and the square of an integral of the process. In detail 



In order to deal with the square of the integral, we define the following state 
process {Xt, Yt) by the following stochastic differential equation (as in Aivaliotis 
and Veretennikov |[T1 or Aivaliotis and Palczewski Gill: 


dXt = b{at, t, Xt) dt -f a{at, t, Xt) dWt, Xq = x 
dYt = f{at, f, Xt) dt, Yo = y. 


(3) 


We will endeavor for existence and uniqueness of solutions to the above SDE. 
The different sets of assumptions, depending on the approach we follow, will 
result in different types of solutions of the above SDE. We will comment on these 
in the relevant sections. Note that / drives the dynamics of Yt in the extended 
state process {Xt, Yt), therefore we will need to impose the same assumptions we 
impose on b. 
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Naturally, we allow the proeess Yt to depend on some initial data Yt^ = y E M. 
Then the value funetion ean be written as v{to, x) := v{tQ, x, 0), where 


v{to,x,y) = sup 
a£A 



- e 


E: 


to,x,y 


{g-{XT,YT)y 


E: 


to,x,y 


{g^{XT,YT)) 


and 

g^{XT,YT):=g{XT,YT,a{-))= [ /(«„ s, X,) ds + = Ft + $(^r). 

Jto 

From now on the dependenee of g on the eontrol will also be ineorporated in the 
expeetation in order to simplify notation. 

For the square of the expeetation, we follow the dual representation = 
sup^g]R{—-0^ — 2'0x} (as in Aivaliotis and Veretennikov [[II). This results to the 
following representation: 


6(t„, I, y) = sup 1 EZ,,,,g(XT, Yt) - e Yt))" 

a€A I 

- sup I - eij^ - 2e^El,^^giXT,YT)] 

^gR *- J 

= sup I y (to, X, y, -0) 

i/,GR ■' 

where y (to, x,?/,-!/;) = sup„g _4 (^(1 - 26»?/;)^(Xt, Ft) - 6»(£/(Vt, Ft))^). 

REMARK 2.1. Note that a similar representation is available for any convex 
function in place of a parabola, although the implementation of this idea may 
be more involved. In the case of an even power function x^", however, such a 
representation has the same level of “complexity” as for x^.' 

x^” = sup + 2n'ilj^^-\x - f)) . 

tpeR 

This may be helpful in studying optimization of a wider family of functionals. 
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3 Viscosity Solutions 

In this section we make the following assumptions: 

(Ay) • The functions a, 6 , /, $ are Borel with respect to (a, t, x) and continu¬ 
ous with respect to (a, x) for every t; moreover, there exist constants 
Ki , K 2 such that 

\\a{a,ti,x) - a{a,t 2 , z)\\ < Ki (Hx - z\\ + |fi - ^ 2 !) 

\\b{a, ti,x) — h{a, t 2 , 2 :)|| < Ki (||a: — z\\ + |fi — ^ 2 !) (Lipschitz condition) 

|/(a,fi,a:) - f{a,t 2 ,z)\ < K 2 (||a: - 2 :|| + \ti - ^ 2 !) 


\\b{a,t,x)\\ < Ki{l + ||x||) 

||a(a,f,a;)|| < Ki[l + || X11) (linear growth condition) 

|/(a,f,x)| < ^2(1 + |lx||) 

For Viscosity solutions we do not need to assume non-degeneracy of matrix aa^. 

Note that the process {Xt, Yt) would have been strongly degenerate even if we had 
assumed non-degeneracy of aa'^. 


THEOREM 3.1. Under assumptions (Ay) for every -0 G M the value function 
V(tQ,x,y) is a unique polynomially growing Viscosity solution of the following 
HJB equation: 


Vt^ -f sup^g^ |6(a,to,a^)^K + \tr[aa'^{a,to-,x)Va,x) + f{u,to,x)Vy^ 
V{T,x,y,'f) = (1 - 2eij)g{x,y) - e{g{x,y)f, 


0 , 

(4) 


Proof We rewrite (01) in a canonical form: 


-Vto{to,X,f) - Hi^to,X,Vs:{to,X,il),Vs,i:{to,x),il) = 0, 

yiT^x^ip) = 0, 

where x = {x,y), the Hamiltonian H is given by 


H(t,{x,y),p,Mj = sup b{u,t,x)^p + -tr(^ay{u,t,x)M^ + pyf{u,t,x) 
^ ^ u£A L 2 
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with p = {p,Py) and M is obtained from M by removing the last row and column. 
Assumptions (Ay) imply that the domain of the Hamiltonian is the whole space 
(dom{H) = {{t,x,p, M) G [0,T] x M"' x M"' x §"■}) and H is continuous. By 
virtue of IfTTl Theorem 4.3.1], y is a viscosity solution of dH) (it is clearly of 
polynomial growth because of the assumptions on the growth of / and <h). Due to 
the Lipschitz continuity in t, x of f,b and a the value function V is continuous at 
the terminal time t = T. Hence, the comparison theorem ( IfTTl Theorem 4.4.5]) 
yields the continuity of V and assures that y is a unique continuous polynomially 
growing viscosity solution to (fU). □ 


4 Sobolev Solutions 

In this section we suggest suitable HJB equations for the suggested mean-variance 
problem, as reformulated in the previous section. The solutions of parabolic HJBs 
will be considered in the Sobolev classes with one derivative with respect 

to t and two with respect to x in Lp. Denote W ’ = np>i For the functions 
of three variables, v{t, x,y),0<t<T, x,y^ R‘^, we will use a similar Sobolev 
class 

Throughout this section, we assume the following 

\s • The functions a,b,f are Borel with respect to {u,t,x), continuous 
with respect to {u, x) and continuous with respect to x uniformly over 
u for each t. <h(x) is continuous with respect to x. Moreover, 


||(t(m, t, x) — a{u, t, x')|| < Ki\x — x'\ 

\\b{u, t, x) — b{u, t, x')|| < Ki\x — x'\ (Lipschitz condition) 

\f{u,t,x) - f{u,t,x')\ < K 2 \x - x'\ 


\\a{u,t,x)\\ + \\b{u,t,x)\\ < K{1 + ||x||) 
\f{u,t,x)\ < K 2{1 + ||x||) 

• |*h(x)| < ^' 2(1 + ||a;||™'). 

• aa^ is uniformly non-degenerate. 


(linear growth condition) 
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In order to establish the existenee of solutions in Sobolev spaees, it is es¬ 
sential that the resulting HJB equations are non-degenerate. However, it is elear 
that the state proeess ([3]) is strongly degenerate and so will be the resulting HJB 
equation for problem dH). In order to avoid degeneraey, apart from assuming non- 
degeneraey for aa^ we add a small eonstant positive diffusion eoeffieient with an 
independent (to Wt) Wiener proeess to the SDE for in (|3]). The regularised state 
proeess beeomes: 


dXt = b{at,t, Xt) dt + a{at, t, Xt) dWt, (5) 

dYf = f{at,t,Xt)dt +eWt-to- (6) 

Aeeordingly we define the regularized value funetion: 


v‘{t„,x,y) = sup ( 
a&A I 

+ sup Yf) ] 

^GR '' 

= sup 

V^gR ^ 

where 

X, ?/, = sup El ({I - 2eij)g{XT, Y^) - Y^)f) . 

THEOREM 4.1. Under assumptions (As) for every '0 G M the value function 

_ ^ 2 2 

X, y) is a unique solution inW ’ ’ of the following HJB equation: 

yf + {b{u, to, x)^Vf + ^tr{aa^{u, to, x)Vf^) + f{u, t, x)Vf + leVfy^ = 0, 

V"{T, X, y, 'f) = {1- 28f)g(x, y) - 9[g{x, y)f. 

(7) 

Proof Under Assumptions (A^) the value funetion has first and seeond order 
bounded generalised derivatives with respeet to spaee and first order bounded gen¬ 
eralised derivative with respeet to time. The rest follows from lUl Chapter 3 and 
4]. □ 
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In the next seetion, we will show that the funetion 1/^’^ is loeally Lipsehitz 
in Ip and grows at most linearly in this variable. Henee, the supremum is again 
attained at some 'p from a elosed interval. Then the external optimisation problem 
beeomes: 

vpto, X, y) = sup [V'^'Pto, X, y, p) - Op"^] . ( 8 ) 

■4’ 

5 Properties of value functions 

In this seetions we show some properties of the value funetions that are eommon 
in both approaehes deseribed above. These are important properties that allow the 
numerieal solution of the mean-variance problem to be tractable. We assume that 
a new set of assumptions (Aq) holds, which is the intersection of the assumptions 
(As) and (Ay). 

THEOREM 5.1. Under Assumptions (Aq): 

i) The functions V, V^, in short) are continuous in p and convex in p. If 
f, $ are non-negative, V, are decreasing in p. 

a) There exists a constant C such that 

\V^^\to,x,y,p) - V^^\to,x,y,p')\ < C (1 + ||x|l) \p - p'\. 

Hi) The value function v is given by 

v{to,x)= sup {V^^\to,x,0,p) - ep"^], 


where 

Pmin — sup Xt, Yf) 

a£A 

and 

Pmax = - inf Ep g{t, Xt, Yt) 
aeA 

Proof The Proof follows the same line of reasoning as in |l2l [Theorem 2.2] and 
making use of the definition for the function g. For part (i), it is straightforward 
to prove convexity, which implies continuity with respect io p. It is clear from 
the definition of V, that they are decreasing in p for non-negative /, $. For 
part (ii) we check that grows at most linearly, which implies that the mapping 
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h{ip) = V^{to, x,y,'ijj) — attains its maximum in a compact interval. By 
convexity, has well-defined directional derivatives. Henee, h also has well- 
defined direetional derivatives and in a point where the maximum is attained the 
left-hand side derivative is non-negative while the right-hand side derivative is 
non-positive. We then show that > 0 for < 'tpmin and d~h{'ijj) < 

0 for ifj > 'tjjmax- This implies that the eonditions for maximum ean only be 
satisfied in the interval [ijjmin, V^max]- We skip further details and refer the reader 
to B [Theorem 2.2]. 

□ 


THEOREM 5.2. Under assumptions (Aq): 


sup \v^{tQ,x,y) - v{tQ,x,y)\ < e^e{T - to). 

t,x,y 


(9) 


Proof. We have, 

\v^{to,x,y) - v{to,x,y)\ < sup\{V%to,x,y,'ilj) - -V{to,x,y,f) + 


= sup I srxpEfig{XT,Yf)[l - 20^] - e{g{XT,Yf)y 

it ^ aeA ^ 


- snpE:fgiXT,YT)[l - 20 ^] - e{g{XT,YT)) 

< sup sup \Ell {g{XT, Yf) - Yt)) [1 - 20^] 

it a(iA '' 

- e({g(XT,Y})f - {g(XT.Y’)) 


= sup sup \E‘ 

it aeA 


-20^] / edWt 

Jto 

9 {g{XT, Yf) - g{XT, 1^)) Yf) + g{XT, Yt)) } | 

-2ef] [ edWt 


= sup sup 

it 06.4 Jto 

-e\2 [ edWt [ fit,Xt)dt 


'to 


'to 


+ i^J edWtj +2$(Xt) / edWt 

<e^e{T-to), 


'to 


( 10 ) 
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since the brownian motion Wt is independent of Xt. This eoneludes the proof. □ 


6 Optimal Strategies 

6.1 Sobolev approach 

When we work with solutions of HJB equations in Sobolev spaces, then there is 
an automatie verifieation theorem that ensures the optimality of the strategy that 
eorresponds to the partieular solution. This eonvenient property is mainly due 
to the availability of some kind of ltd formula (in this ease Ito-Krylov formula). 
Since, however the problem solved is regularised, we ean only hope for “almost- 
optimal” strategies for the original problem. 

DEFINITION 6.1. Let e > 0. A strategy a E Ais said to be e—optimal for {t, x) 
ifv{t, x) < v°‘(t, x) + e, where v{t, x) = supQ,g _4 x). 

LEMMA 6.2. Under assumptions (Aq), for any strategy a E A, we have the 
following bounds 

- u“(to,a:,|/)| < Ce^. (11) 

with C = 0{T — to). 

Proof. One ean show similarly to Theorem 15.21 that 

\v^''^{to,x,y) - 'u“(fo,a:,2/)| < e^e{T - to), 


therefore 


X, y) - Ce'^ < v°‘{to, x, y) < x, y) + Ce‘^. (12) 

□ 

THEOREM 6.3. Assume (Aq). Let the strategy E Abe the optimal strategy 
for the problem (IS]), or if the supremum is not attained, let the strategy E Abe a 
5—optimal strategy for the same problem. Then, the same strategy is e-optimalfor 
the original degenerate value function with appropriate choice of the constant e. 

Proof. Suppose that the value funetion of the degenerate problem attains its supre¬ 
mum for a strategy a. Then, we would have from Lemma [6]2] 

'f;“(fo, X, y) - Ce^ < x, y) < v°‘{to, x, y) + Ce^. (13) 
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Furthermore, beeause the strategy is optimal for the regularised value funetion 
(assuming that the supremum is attained), we know that 

v^’^\to,x,y) > (14) 


So, 

v°‘{to,x,y) - Ce^ < v^’'^{to,x,y) < v^’°‘\to,x,y) < v'^\to,x,y) + Ce"^. (15) 


Henee 

v'^{to,x,y) - 2Ce^ < v'^\to,x,y), (16) 

i.e. is 2C'£^—optimal for r;" or e = 2(7^^ in this ease. 

In the ease that the degenerate value funetion does not attain a supremum, a 
7 — optimal strategy exists, namely a. Then 

v'^{to,x,y) >snpv'^{to,x,y)--f (17) 

a^A 

and due to the bounds (fTTl) and inequality (fTTI) we have 

supu“(fo,x,|/)- 7 -C£^ < v'"{to,x,y)-Ce'^ < v^’°‘{to,x,y) < v°'{to,x,y)+Ce'^. 
a€A 

(18) 

Now, suppose again that (to, x,y) = suPq ,£_4 n^’"(to, !/)• That means 

v^’^\to,x,y) > v^’^{to,x,y). 

So, 


sup u“(fo, x,y)-'y- Ce^ < u"(fo, x, y) - Ce^ < x, y) 

aeA 

< u^’“"(fo,a:,|/) < v°‘\to,x,y) + Ce^. (19) 

Thus, 

sup u“(fo, X, y) < (to, X, I/) + 7 + 2Ce^, (20) 

a&A 

i.e. is an (7 + 2 Ce^)—optimal strategy for u“(fo, x, ?/) or e = 7 + 2Ce^ in this 
ease. 

Finally, in the ease that supQ,g _4 u^’“(fo, x, y) is not attained, a 5—optimal strat¬ 
egy is attained instead, sueh that u^’“^(fo, x, y) < sup„g _4 u^’"(fo, x, |/) — 5. 
Following the same reasoning as previously (note that a is 7 —optimal for u^), we 
get 


sup u“(fo, x,y) — — 5 — Ce^ < v°‘{to, x, y) — Ce^ — 5 

a<^A 

< x,y)-5 < (to, X, y) < u"" (fo, x, y) + Ce'^. (21) 
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Therefore, 


( 22 ) 


snpv°'{to,x,y) < v°'\tQ,x,y) + 7 + 5 + 2C'£^, 
a&A 

i.e. of is an (7 + 5 + 2C'£^)—optimal strategy for v^{to,x,y) or e = 7 + 5 + 
2Ce^. □ 

COROLLARY 6.4. Let assumptions (Aq) hold. The supremum attained for the 
degenerate value function by using Markov strategies is almost the same as the 
one attained in the class of admissible strategies. So Markov strategies are e— 
optimal for the degenerate value function. 

Proof. Since we deal everywhere with "first moment theory" and additionally an 
optimal f can always be found in a real interval, we know that Markov strategies 
are sufficient for the problem (see DTI). Therefore, because of the previous The¬ 
orem, Markov strategies are sufficient in obtaining an e—optimal strategy for the 
degenerate value function. □ 

6.2 Viscosity Approach 

When working with viscosity solutions, there is no verification theorem that can 
be applied under the assumptions made in this paper (see Gozzi et al. (bl for the 
latest results on the verification theorem for viscosity solutions). One possible ap¬ 
proach is to verify the optimality of the strategies using Monte Carlo simulations 
using the strategy calculated from the solution of the HJB equation and compare 
the value of the value function obtained by simulation to the one from the numer¬ 
ical scheme. 


7 Concluding Remarks 

In this paper, we formulated a general mean-variance problem in continuous time 
that includes a functional with two terms: an integral that depends on the whole 
trajectory of the controlled process and a terminal time one. We interpreted the 
problem first as a terminal time problem, through the introduction of a coupled 
state process with an additional dimension and then transformed it into a superpo¬ 
sition of a static and a dynamic optimization problem, where the latter is feasible 
for dynamic programming methods and for which we were able to write down an 
HJB equation. We proved existence and uniqueness of solutions both in viscosity 
sense and also in classical (Sobolev) sense. The advantage of the first approach is 
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that there is no need to address the inherent degeneracy of the coupled state pro¬ 
cess, whereas numerical solutions can be employed to solve the problem and to 
even show optimality thought Monte Carlo simulations; however, in this approach 
a verification theorem is not readily applicable under the assumptions we use. 

When following the Sobolev approach, a regularisation of the state process 
is required. This has the advantage that a verification theorem can be obtained 
through Ito-Krylov’s formula. We then showed that strategies obtained through 
this route are e—optimal. 

Finally recall that using the hint shown in the Remark [ZTl the results in both 
sections[3]and|4]allow extentions to the case of functionals that involve even power 

functions like /(a^, s, ds + <I>(Xt))) . 
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